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Abstract 


1.  Introduction 


The  Engineer  Research  and  Development  Center 
Major  Shared  Resource  Center  (ERDC  MSRC)  has  been 
tasked  by  the  Department  of  Defense  (DoD)  High 
Performance  Computing  Modernization  Program 
(HP CMP)  to  adopt  a program-wide  design  for  the 
development  of  the  recently  introduced  ezHPC  user 
interface.  The  purpose  of  ezHPC  is  to  provide  all 
HPCMP  users  intuitive,  efficient  access  to  high 
performance  computing  (HPC)  resources  needed  in  their 
regular  course  of  work.  At  the  core  of  user-centered 
design  is  an  emphasis  on  fully  understanding  HPC  users 
and  their  work  environment,  enabling  support  for  user 
needs,  not  developer  concerns,  to  drive  the  design 
process.  Fulfilling  this  goal  requires  the  use  of  a variety 
of  methods  from  the  field  of  usability  engineering.  For 
ezHPC,  methods  have  included  to  date  (1)  inclusion  of 
users  as  part  of  the  design  team,  (2)  a facilitated  focus 
group  session,  (3)  a usability  walkthrough,  (4)  a usability 
evaluation,  and  (5)  rapid  prototyping. 

In  adherence  to  user-centered  design,  a user  group 
(UG)  was  formed  to  serve  as  participants  in  the  design  of 
the  ezHPC  software.  ezHPC  UG  members  participated  in 
a recent  workshop  .for  the  purpose  of  assessing  and 
improving  the  usability  of  the  program-wide  version  of 
ezHPC. 

Results  are  currently  being  used  to  guide  the  new 
design  of  the  ezHPC  interface  using  rapid  prototyping. 
Prototypes  are  under  development  for  regular  reviews  by 
the  UG,  followed  by  iterative  refinement.  This  direct 
early  involvement  of  users  ensures  more  responsive,  user- 
centered  interfaces.  Formal,  large-scale  usability 
evaluations  are  expected  in  the  latter  half  of  the  software 
cycle. 


For  ezHPC,  usability  methods  have  included  to  date 
(1)  inclusion  of  users  as  part  of  the  design  team,  (2)  a 
facilitated  focus  group  session,  (3)  a usability 
walkthrough,  (4)  a usability  evaluation,  and  (5)  rapid 
prototyping. 

In  adherence  to  user-centered  design,  a user  group 
(UG)  was  formed  to  serve  as  participants  in  the  design  of 
the  ezHPC  software.  ezHPC  UG  members  participated  in 
a recent  workshop  for  the  purpose  of  assessing  and 
improving  the  usability  of  the  program-wide  version  of 
ezHPC. 

Results  are  currently  being  used  to  guide  the  new 
design  of  the  ezHPC  interface  using  rapid  prototyping. 
This  paper  reports  results  and  status  of  usability-related 
activities  conducted  to  date  and  concludes  with  future 
plans  for  the  project. 

2.  Usability  Workshop 

A total  of  seven  HPC  users  from  a variety  of 
backgrounds  participated  in  the  workshop.  Though  this 
number  may  seem  small,  studies  have  shown  that,  if 
properly  composed,  a group  of  this  size  is  optimal  for 
conducting  a user-centered  design  (UCD)  process1'1. 
Such  a process  diverges  from  conventional  software 
development  methods  in  that  it  focuses  on  eliciting  user 
needs,  rather  than  developer  concerns,  to  drive  the  design. 
(Extensive  usability  testing  occurs  later  in  the 
development  cycle  and  involves  larger  numbers  of  users, 
but  this  is  both  impractical  and  unnecessary  in  the  design 
stage.)  Hence,  to  support  the  user-centered  design 
process,  the  workshop  participants  were  carefully  chosen 
to  provide  the  broadest  possible  window  into  typical  user 
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needs.  Out  of  the  seven  participants,  four  reported 
spending  at  least  50  percent  or  more  of  their  time  doing 
direct  technical  work;  the  other  three  participants 
provided  user  technical  support  to  a large  group  of  HPC 
users,  each  from  different  agencies  with  somewhat 
differing  user  cultures;  e.g.,  some  involved  the  use  of 
more  students  than  others.  The  latter  point  regarding 
differing  user  cultures  is  significant  in  obtaining  the 
broadest  representation  of  the  user  population. 

2.1.  Preworkshop  Data 

Prior  to  the  workshop,  each  participant  replied  to  a 
brief  e-mail  survey  with  the  purpose  of  eliciting  a broad 
view  of  the  typical  user  workflow.  These  data  were 
studied  prior  to  the  workshop  and  used  to  direct  a focus 
group  session  held  the  first  morning.  A diagram  depicting 
a high-level  workflow,  based  on  these  data,  is  shown  in 
Figure  1.  An  analysis  of  responses  to  the  survey  revealed 
three  basic  categories  of  work  activities:  (1)  performing 
direct  technical  work,  (2)  communicating,  and  (3) 
monitoring.  Specific  activities  and  ranges  for  percentages 
of  time  were  reported  under  each  category.  Note  that 
some  higher  level  activities,  such  as  user  support  and 
supervising,  encompass  more  than  one  of  the  fundamental 
categories.  Thus,  they  are  listed  across  the  appropriate 
categories,  e.g.,  user  support,  which  requires  both 
communicating  and  direct  technical  analysis. 
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Figure  1.  High-level  workflow  diagram 


2.2.  Workshop  Focus  Group  Data 

Focus  groups,  long  used  in  marketing,  have  also 
proven  useful  in  gathering  usability  requirements  if 
properly  facilitated  and  used  in  conjunction  with  other 
data-gathering  techniques121.  The  workflow  analysis 
shown  in  Figure  1 was  used  to  initiate  a focus  group 
session  held  the  first  morning  of  the  workshop. 
Participants  engaged  in  discussion  to  further  refine  the 
understanding  of  the  typical  user  work  environment  and, 
particularly,  the  most  critical  problems  faced. 

Though  the  fundamental  workflow  did  not 
significantly  change,  a focus  emerged  in  a few  key  areas: 
(1)  resource  and  allocation  monitoring,  (2) 
communicating  with  management,  and  (3)  mentoring  new 
employees.  First,  information  that  could  be  used  to 
reduce  turnaround  time  for  getting  jobs  through  queues 


emerged  as  a critical  concern.  Second,  a need  to  convey 
resource  allocation  problems  to  nontechnical  management 
was  strongly  voiced.  Third,  issues  regarding  training  and 
mentoring  new  employees  and  students,  many  of  whom 
may  be  non-native  speakers,  emerged  as  an  area  of  some 
importance. 

2.3.  Usability  Evaluation 

In  addition  to  the  e-mail  survey,  a usability 
walkthrough  was  conducted  by  the  design  and 
development  team  prior  to  the  workshop.  The  usability 
walkthrough  is  a technique  for  identifying  potential 
usability  problems  in  an  interface121.  It  entails  a team  of 
usability  specialists  (and  can  include  developers)  who 
“walk  through”  a set  of  interactive  exercises  with  an 
interface  to  identify  potential  problems,  based  on  known 
usability  heuristics  and  principles13'51.  This  technique  has 
been  reported  to  identify  a significant  percentage  of 
usability  problems  in  the  design  stage.  Results  are 
intended  to  be  used  as  a guide,  however,  not  a 
replacement  for  user  testing.  Hence,  the  results  of  the 
walkthrough  were  used  in  developing  a set  of  tasks  and  an 
accompanying  questionnaire  for  a usability  evaluation 
performed  by  users  at  the  workshop. 

At  the  completion  of  the  morning  focus  group 
session,  participants  engaged  in  a usability  evaluation,  in 
which  they  performed  a variety  of  tasks  using  the  current 
ezHPC  interface  and  then  evaluated  its  ease  of  use  via  a 
questionnaire.  The  tasks  were  designed  to  exercise  all 
critical  aspects  of  the  system  interface  and  to  cover  the 
broadest  possible  range  of  activities  a user  would  perform 
in  a typical  day  or  week,  again,  based  on  results  of  the  e- 
mail  survey,  other  previous  UG  input,  and  the  usability 
walkthrough  results.  In  addition,  emphasis  was  placed  on 
writing  the  tasks  in  user  (not  programmer)  language,  and 
with  maximum  neutrality,  i.e.,  instructing  *what*  task  to 
perform,  but  not  *how*  to  use  the  interface  to  perform  the 
task.  An  example  would  be  as  follows:  “Copy  a file  from 
your  laptop  to  another  HPC  computer,”  versus  “Go  to  the 
file  management  page,  select  the  file  you  want  to  copy.” 
The  latter  phrasing  is  better  suited  for  testing  to  identify 
programming  errors,  not  usability  issues.  The  neutrality 
of  the  first  task  phrasing,  however,  is  key  to  obtaining 
meaningful  usability  assessments161. 

After  performing  these  tasks,  participants  responded 
to  a usability  questionnaire,  which  elicited  two  types  of 
feedback:  (1)  Likert  Scale  ratings  of  system  features171, 
(e.g.,  1-5,  Strongly  Agree  to  Strongly  Disagree),  and  (2) 
freestyle  comments.  Each  type  of  data,  Likert  scale 
ratings,  and  freestyle  comments  (as  well  as  any  other  user 
input)  must  be  considered  to  gain  insight  into  how  to 
improve  the  interface  design.  Highlights  of  results  in  job 
management,  file  management,  script  management,  and 
Web  design  style  are  summarized.  Since  file 
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management  issues  emerged  as  one  of  the  higher  priority 
items,  results  for  this  feature  are  then  presented  in  greater 
detail. 

2.4.  Usability  Evaluation  Results 

In  the  area  of  job  management , results  indicated 
users  required  better  visual  tools  for  job  scheduling,  clear 
history  mechanisms  for  job  queries,  and  unambiguous 
language  for  novice  users  regarding  system  events.  In 
script  management,  users  needed  more  comprehensive, 
easier-to-use  methods  for  finding  existing  scripts,  and 
more  user-centered  methods  for  sharing  scripts.  In  the 
area  of  file  management,  users  required  more  efficient 
methods  for  finding  existing  files,  easier-to-use  methods 
for  copying  files  from  host  to  host  and  from  laptop  to 
host.  Finally  regarding  Web  preferences,  users  preferred 
lighter,  simpler  interaction  styles. 

Again,  the  file  management  interface  features 
sparked  a significant  share  of  the  workshop  discussion, 
not  only  in  the  questionnaire  freestyle  comments  but 
throughout  the  first  day’s  sessions.  These  features  also 
stimulated  some  of  the  most  highly  split  ratings  (positive 
and  negative),  flagging  it  as  a priority  in  the  redesign.  A 
screenshot  of  the  interface  for  the  file  management 
interface  evaluated  is  shown  in  Figure  2. 
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Figure  2.  File  management  screen 


To  reiterate,  this  interface  was  designed  by  a skilled, 
experienced  programming  team,  though  untrained  in 
usability.  Based  on  their  best  understanding  about  users’ 
experience  with  software  tools  and  interfaces,  they  made 
many  seemingly  logical  choices  in  the  interface  paradigm 
and  features.  For  example,  the  dual-pane  paradigm  draws 
on  a common  graphical  interface  for  file  transfer181. 
However,  the  original  intent  of  this  paradigm  was 
primarily  to  support  the  transfer  operation  on  a file,  and 
not  the  many  other  operations  a user  may  wish  to  apply  to 
files,  e.g.,  editing,  listing,  changing  permissions,  and 
searching.  As  an  interface  paradigm,  it  does  not  come 
without  cost:  by  splitting  the  screen  in  half  horizontally,  it 
reduces  the  display  area  for  the  file  listing,  requiring  users 


to  scroll  right  to  view  important  information  about  a file. 
It  also  forces  a left-right  sequencing  that  is  not  intuitive 
for  all  operations,  such  as  changing  permissions  or 
deleting  a file.  Further,  the  interface  used  the  classic 
terminology  for  file  transfers  “upload/download”,  mixing 
a “vertical”  logical  view  (up/  down)  with  a left-right 
interaction  orientation. 

Second,  a “filter”  option  provided  in  the  file  selection 
window  presents  another  example  where  the  developers 
reasonably  relied  on  another  interface  feature  with  which 
their  experienced  UNIX  users  would  be  familiar,  the 
UNIX  filter  command  line  function.  However,  this  does 
not  translate  easily  to  the  graphical,  Web-based  paradigm 
for  several  reasons.  Experienced  UNIX  users, 
knowledgeable  of  the  many  ways  a filter  can  be  applied, 
might  be  unsure  which  was  offered,  while  inexperienced 
users  would  likely  be  unfamiliar  with  the  term  or 
technique.  These  issues  did  not  render  this  feature 
inherently  bad  or  good;  they  simply  underlined  the 
necessity  of  determining  the  critical  underlying  need  for 
all  users  and  designing  an  approach  to  fit  that  need. 
These  potential  problems  had  been  identified  in  the 
usability  walkthrough;  the  evaluation  yielded  further 
insight. 

Listing  and  Viewing.  The  most  positive  Likert 
ratings  pertained  to  ease  of  obtaining  a file  listing  (5 
Agree,  1 Neutral,  and  1 Disagree  to  “It  was  easy  to  obtain 
a file  listng.”).  Despite  the  overall  positive  ratings, 
several  user  comments  related  to  this  feature,  particularly 
file  filtering,  were  negative  and  important  to  note  for  the 
redesign.  Excerpted  comments  include 

• “When  filtering  file  listing,  I had  no  idea  how  the 
filtering  was  done:  whether  the  filter  was  applied 
to  the  file  name,  the  full  path,  ownership,  etc.” 
(User#l) 

• “I  found  the  filtering  mechanism  confusing.” 
(User  #4) 

• ”Our  new  users  are  not  familiar  with  filtering.” 
(User  #7) 

Copying  and  Transferring.  The  most  split  Likert 
scale  ratings,  as  well  as  some  of  the  most  negative 
freestyle  comments,  pertained  to  file  transfers  and  copies. 
A near  equal  split  occurred  on  user’s  ratings  of  the  ease  of 
copying  files  from  one  host  to  another  (three 
agree/strongly  agree,  and  two  disagree),  as  shown  in 
Figure  3.  An  exactly  equal  split  of  user  ratings  occurred 
for  the  question  on  ease  of  transfers  from  laptops  to  hosts 
(three  participants  agreed,  three  disagreed),  as  shown  in 
Figure  4. 
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Figure  3.  Results  for  ease  of  copy  from  host  to  host 


Figure  4.  Results  for  ease  of  copy  from  laptop  to  host 


Split  ratings  often  indicate  problem  areas,  but  require 
more  analysis  of  user  comments  than  more  consistent 
ratings.  Difficulties  encountered  included,  first, 
understanding  of  how  to  actually  perform  the  transfer  or 
copy,  and  second,  knowing  whether  the  transfer  had 
occurred.  Excerpted  comments  include 

• “I  do  not  like  the  upload  and  download  buttons. 
They  are  confusing.  I think  the  local  client 
should  be  selectable  in  the  left  or  right 
window...”  (User  #3) 

• “Problems  transferring  large  files  from  PC  to 

HPC.  Need  better  upload/download 

management.  Can’t  tell  when  file  is  transferred, 
would  like  visual  feedback. . .”  (User  #7) 

Finally,  two  questions  in  the  usability  questionnaire 
pertained  to  user  preferences  and  habits  regarding  file 
management.  Responses  showed  overall  (six  out  of  seven 
participants)  that  finding  files  and  copying  files  were  both 
the  most  frequent  and  most  critical  file  management  tasks 
they  performed  on  a daily  basis.  Freestyle  comments  on 
this  feature,  along  with  post-assessment  verbal  discussion, 
indicated  that  what  users  needed  most  critically  was  an 
ability  to  find  files  through  searching  the  file  system, 
rather  than  applying  filters  to  specific  listings  or  file 


system  locations.  Though  they  rated  the  latter  as 
desirable,  they  viewed  a general  search  capability  as  a 
higher  priority  fundamental  need. 

3.  Prototyping 

Prototyping  is  a critical  phase  in  user-centered  design 
and  involves  iteratively  generating  interface  designs  and 
refining  them  based  on  user  feedback191.  Unlike  the 
functional  aspects  of  software,  user  interface  features 
demand  this  level  of  interaction  with  users  to  achieve 
correct  results.  The  team  began  prototyping  the  new 
interface  design  once  feedback  was  given  on  the 
prioritized  usability  results.  A mixture  of  horizontal 
(overall  system  organization)  and  vertical  (specific 
feature)  prototyping  is  being  used[l0].  Highlights  of 
prototyping  results  to  date  are  given  in  the  following 
subsections. 

3.1.  File  Management 

The  home  page  and  file  management  pages  were 
mocked  up  for  user  review.  Only  the  file  management 
screen  mockups  are  shown  in  Figures  5 and  6,  since  they 
show  critical  refinements  to  the  home  page,  e.g.,  a lighter 
interaction  style  and  streamlined  menu.  Key  features  of 
the  file  management  redesign  include  (1)  a single-pane 
approach  with  full  horizontal  file  listing  as  the  default,  (2) 
a redesigned  set  of  file  operations,  newly  named  and 
placed  in  one  location  across  the  top  of  the  single  pane, 
(3)  a file  search  capability,  and  (4)  a redesigned  copy 
interaction. 

The  single-pane  approach  allows  a user  to  view  a full 
detailed  file  listing  by  default  and  reduces  scrolling  and 
other  readability  issues  in  the  original  interaction.  The 
operations  supported  on  a file,  as  well  as  their  names, 
were  also  redesigned.  For  example,  the 
“upload/download”  and  “transfer”  terminology  was 
discarded  in  favor  of  a simple  “copy”  operation  that  could 
be  applied  to  any  file,  regardless  of  its  location. 

Some  names  for  file  operations  were  retained, 
however,  such  as  “chmod”  for  changing  file  permissions. 
This  term,  understood  by  experienced  UNIX  users,  was 
an  example  where  their  needs  and  those  of  novice  Unix 
users  required  a balance.  For  new  users,  however,  once 
learned,  this  terminology  is  unlikely  to  change.  Also, 
training  support  will  be  included  through  such  means  as 
documentation,  tool  tips,  and  a dialog  interaction  labeled 
“Change  File  Permissions”  that  will  occur  when  the 
button  is  pressed. 

Second,  in  response  to  the  need  expressed  by  users 
for  the  ability  to  search  the  file  system,  a new  operation, 
“Find  Files”,  was  added.  This  allows  users  to  perform  a 
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search  of  the  file  system,  based  on  keywords  they 
provide. 

Finally,  the  interaction  for  the  copy  operation  was 
redesigned  to  use  a dialog,  prompting  users  for  source  and 
destination  files  and  locations.  A progress  meter 
displayed  on  the  bottom  of  the  screen  indicates  the  status 
of  the  copy  operation.  This  bar  can  be  minimized  or 
dismissed. 

User  reaction  to  the  mockups  has  been  positive 
overall  with  some  mixed  reviews  on  the  copy  operation. 
Several  users  expressed  the  desire  to  provide  persistent 
access  to  the  destination  screen  in  the  interaction. 
Methods  for  providing  this  while  retaining  the  single-pane 
default  view  are  under  consideration  by  the  design  team. 


4.  Conclusions 

This  paper  has  presented  ongoing  results  of  applying 
a user-centered  design  process  to  a program-wide  version 
of  the  ezHPC  user  interface.  This  process  has  applied 
several  well-tested  usability  techniques,  most  notably  the 
formation  and  input  of  a user  advisory  group  in  activities 
including  a focus  group  session,  a usability  evaluation, 
and  rapid  prototyping. 

Currently,  the  prototypes  are  incrementally  evolving 
in  response  to  feedback  from  the  user  group  to  a final 
system;  summative  usability  and  error  testing  is  scheduled 
in  the  latter  half  of  the  project  life  cycle. 
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Figure  7.  File  management  mockup 
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Figure  8.  File  copy  interaction  mockup 
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